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Abstract 

We demonstrate the advantages of feedforward 
loops using a Boolean network, which is one of 
the discrete dynamical models for transcriptional 
regulatory networks. After comparing the dynamical 
behaviors of network embedded feedback and feed- 
forward loops, we found that feedforward loops can 
provide higher temporal order (coherence) with lower 
entropy (randomness) in a temporal program of gene 
expression. In addition, complexity of the state space 
that increases with longer length of attractors and 
greater number of attractors is also reduced for net- 
works with more feedforward loops. Feedback loops 
show opposite effects on dynamics of the networks. 
These results suggest that feedforward loops are one 
of the favorable local structures in biomolecular and 
neuronal networks. 



and edges among the nodes which are found to 
be statistically significant, and can be regarded as 
functional modules [HI |20l [22] . Since feedback and 
feedforward loops are motif structures as well as basic 
and ubiquitous circuits in man-made systems, one can 
expect that transcriptional regulatory networks also 
have both feedback and feedforward loops; however, 
only the feedforward loops prevail [181 IlOl [22] ■ Other 
biological networks such as signal transduction and 
neuronal networks also have similar tendencies, which 
suggests that feedforward loops are favored in complex 
biological networks. In general, although massive 
available network data demonstrate the statistical 
significance, it is unclear why feedforward loops 
are advantageous over feedback loops in dynamical 
systems. 
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2 Model and method 
2.1 Boolean network 



1 Introduction 

Recent studies of natural complex networks [17j 
including transcriptional regulatory networks in cells 
have revealed at least three stastistical properties: 
long-tailed global connectivity distributions having a 
small number of highly connected nodes; small-world 
properties of short path lengths between any two 
nodes; and highly clustered connections among 
adjacent nodes [2l[Il[8l[S[18l[20l[22l|2l]- The last 
local structures called motifs consist of a few nodes 
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The dynamics of the Boolean networks [6] [21] is 
determined by the equation 

X,it + l) = B,[X{t)] (z-l,2,...,iV), (1) 

where X{t) is a binary state, either or 1, of node 
i at time t, Bi{-) are Boolean functions [See Table [1] 
used to simultaneously update the state of node z, and 
X{t) is a binary vector that gives the states of the 
N nodes in the network. After assigning the initial 
states X{Q) to the nodes, the successive states of the 
nodes are updated by input states and their Boolean 
functions. The dynamical behavior of these networks 



is represented by a time series of binary states. The 
time course follows a transient phase from an initial 
state until a periodic pattern, called an attractor, is 
established [See Fig. [3]. 

Table 1: 4 of 16 Boolean functions with indegree Kin 
— 2. In this paper, we used Boolean functions shown 
below, because of the feasibility of computation and 
the biological meaning of the functions [111 [25] . 
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2.2 Numerical condition 

To investigate the effects of feedback and feedfor- 
ward loops on the dyamics Boolean networks, we con- 
structed many networks with varying numbers of in- 
dependent feedback or feedforward loops, where both 
loops consisted of three nodes and three directed edges 
[See Fig. [Tjand Table [2]. After embedding the spec- 
ified number of loops, the rest of the directed edges 
were assigned at random. 

In total, we constructed 9 xlO'* Boolean networks 
[See Table [2] with fixed a network size. We applied 
2 xlO'^ initial states to each network. Four different 
Boolean functions [See Table [T] were used in the same 
frequency [See Table [1]. 



Table 2: Numerical condition: All networks consist of 
the same amount of network resources, nodes, directed 
edges, and the number of Boolean functions. The dif- 
ference among the generated networks lies in the style 
of the connections. 
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Figure 1: Two embedded structures: FFL and FBL 
stand for feedforward loop and feedback loop, respec- 
tively. The loops consist of 3 nodes (squares) and 3 
directed edges. 



2.3 Entropy and mutual information 

We measured the entropy (randomness) and mu- 
tual information (coherence) of state variables to char- 
acterize the temporal (series) structure of state vari- 
ables in the Boolean networks [21 [H [51 [5] [See Fig. [5]. 




Figure 2: Flow of state variables from upstream nodes 
to downstream. Since the input connectivity for all 
nodes is 2 {—Ki„), there are two pathways for mutual 
information in each node. The input sequence Inn for 
the node i is the same as the output sequence of an up- 
stream node Outji, and the output sequence of Outi 
for the node i is the same as the input sequence of a 
downstream node Ink ■ When node i has multiple out- 
put connections, they have the same binary sequence 
because state variables in networks are subject to Eq. 

CD. 



Size of networks N 


128 nodes 


Connectivity 


For all nodes. Kin = Ko^t = 2 


Boolean function 


only AND type [See Table [1] 


Types of loop 


FFL, FBL [See Fig. [l] 


Number of loops 


0, 10, 20, 30, and 40 


Number of edges 


256 


Number of initial states 


2000 per network 


Number of realizations 


10^ in each condition 



2.4 Complexity of state space 

To characterize the complexity of state space from 
initial states [See Fig. [3], we use two measures |19i : 

1. Sum of length of attractors: Each network may 
contain different number of attractors and the 
lengths of the attractors may also vary. The mea- 
sures define the total length of the attractors in 
state space. 



2. Basin entropy [See Appendix 15. II for details]: 

H Basin = - ^ P{i) l0g2 p{i) (2) 

i 

where, ^iP{i) — 1- The two measures indicate the 
complexity of state space from initial states. Accord- 
ing to the definitions, the larger values of two char- 
acteristics sgnify higher complexity of the basin of at- 
traction [See Fig. [3]. 
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Figure 3: Example of a state space: There are differ- 
ent 2^ states (shown as circles) in the space for each 
network. The numbers 3 and 7 inside attractors indi- 
cate the lengths of the attractors. The numbers also 
correspond to the size of the basin of attraction. 



2.5 Path length 

To obtain the structual properties of propagating 
pathway of the state variables, we measured two prop- 
erties [iiiniiniiii]: 

1. Path length, which is the average number directed 
edges in the shortest path from a node to all 
reachable nodes. 

2. Average path length, which is the average number 
of the path lengths for all the nodes. 



3 Results 

In total, we obtained 5479157 attractors from 9 
xlO"* networks with 1.8 xlO* intial states. The size 
of entropy [See Fig. [7], mutual information [See Fig. 
[8], total length of attractors [See Fig. [6], and basin 
of entropy [See Fig. [7] are measured from the attrac- 
tors. We also obtained the dependence of average path 
length on the number of embedded loops [See Fig. [S]. 
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The specified number of 
embedded loops 



Figure 4: Relationship between the specified number 
of embedded loops and the actual number of embedded 
loops. Symbols indicate mean ± SD. Dashed line is 
given hy y = X. 



Figure [3] shows that our successful method for em- 
bedding loops in Boolean networks. The average path 
lengths increases with increasing number of embed- 
ded loops [See Fig. [5] and see Appendix 15. 2j . Fig- 




The specified number of 
embedded loops 



Figure 5: Relationship between the number of embed- 
ded loops and average path length. Symbols indicate 
mean ± SD. 



ure [H] shows the complexity of state-space structures. 
Feedforward and feedback loops have opposite effects. 
Please note that both the number of attractors and the 
sum of the lengths of attractors with feedforward loops 
slightly decrease. Both entropies in Fig. [7]indicate op- 
posite effects on different loop structures. Unlike Figs. 
[B] and [3 the size of mutual information increases with 




1000 ra 
H800 "5 



o 

E 



The specified number of 
embedded loops 



Figure 6: Relationship between the number of embed- 
ded loops and sum of length of attractors (upper), and 
the number of attractors (bottom). Symbols indicate 
mean values. 



increasing number of embedding loops [Fig. [S]. 



4 Summary and discussion 

We examined the effects of embedding loops on 
dynamical and topological properties. The networks 
with more feedback as well as feedforward loops ex- 
hibit longer path lengths. As for the dynamical prop- 
erties, the networks with more feedback loops show a 
larger number of attractors and greater size of entropy 
and mutual information, demonstrating that feedback 
loops increase the complexity of the state space of the 
networks. In other words, the feedback loops behave as 
pattern generators of a temporal program of gene ex- 
pression or entropy generators, and the resultant mu- 
tual information (coherence) is driven by the entropy. 
On the other hand, the networks with more feedfor- 
ward loops show a smaller number of attractors and 
size of entropy but larger mutual information. The 
effects indicate that feedforward loops play a role of 
stabilizing the state space as well as organizing tem- 
poral patterns with less entropy. 

The structure of feedback loops resembles that of 
a repressilator (TB], which is a synthetic genetic reg- 
ulatory network consisting of three genes connected 
in a feedback loop. The repressilator shows self- 
sustained oscillations reminiscent of our results from 
Boolean networks with feedback loops. Other studies 
[l3t [22l [24] demonstrate that feedforward loops can 
exhibit temporal and spatial order with differential 



equations. Our results are similar, as shown in Fig. 

m 

Control parameters for the dynamics of Boolean 
networks are input connectivity. Km, the size of net- 
work, the bias of Boolean functions, and output con- 
nectivity distributions [U [151 13 ■ In this report, 
we change only the connection style with the same 
amount of network resources [See Table [2] . Neverthe- 
less, Figs. [6] - [8| demonstrate that the internal con- 
nection style may well be regarded as a novel control 
parameter for the dynamics of Boolean networks. 

Our results may provide a blueprint for the design 
of an artificial regulatory gene network [161 US] , elu- 
cidate the role of loop structures in dynamical sys- 
tems, and provide some insight into the prediction of 
relationships between complex network structures, be- 
haviors, and functions. Since the currently available 
biological network data show resultant structures af- 
ter evolutionary and/or developmental processes, our 
constructive approach [H [H [H [TOl El [12] is one of the 
promising ways for disentangling natural large-scale 
complex networks. 
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5 Appendix 

5.1 Basin entropy 

The definition of basin entropy deals with the de- 
gree of partitioning of state space by basin of attrac- 
tions. For example, even though the same number of 
attractors may be found by starting from different ini- 
tal states, the portions of the number of initial states 
that follow into those attractors can vary from one 
network to the other. Eq. ([2|) gives the minimum and 
maximum entropy when all 2000 different initial states 
fall into only one attractor and 2000 different attrac- 
tors, respectively. By the definition, p{i) in Eq. ([2]) is 
giving as 



p{i) 



2000 



Z^ 



a, = 2000 



(3) 
(4) 



where a^ is the number of initial states that reached 
the i-th attractor, The number 2000 is obtained the 
numerical condition [See Table H] . 



5.2 Average path length 

Since we generated networks with random assign- 
ment of edges between nodes under fixed connectivity, 
Kin = Kout = 2, we can assume a tree-like structure of 
a nodes in a network (Fig. [9K). With this assumption, 
it can be said that the number of downstream nodes 
increases exponentially, and the relationships between 
average path length L and network size N can be ex- 
pressed as follows : 



2(2^ 



11 



N - 1 



(5) 



The average pathlength L can be obtained by trans- 
forming Eq. ([5]), 



log2 



N -1 



(6) 



When the number of embedded loops increases, the 
average actual number of downstream nodes decreases 
(Fig. [9)d), leading to the reduction of the common 
ratio in Eq. ([5]). Using the reduced common ratio x 
where x takes 1 < a; < 2, Eq. ^ changes to 



2(a:^ - 1) 
a;i -1 



= iV-l. 



Similarly, Eq. ([7]) can be transformed into 

\{x-l){N-l) 
L = log^ 



(7) 



(8) 




The specified number of 
embedded loops 



Figure 7: Relationship between the number of em- 
bedded loops and size of entropy (upper), and basin 
entropy (bottom). Symbols indicate mean values. 




The specified number of 
embedded loops 



Figure 8: Relationship between the number of embed- 
ded loops and the size of mutial information. Symbols 
indicate mean values. 
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Figure 9: Differences in the number of downstream 
nodes: Daslied arrows indicate output connection from 
a node a) and from loops b). a) The number of down- 
stream nodes increases almost exponentially with ran- 
dom assignment of edges, b) With embedding loops, 
the average number of downstream nodes decreases 
since the number of edges inside the loops becomes 
large. 



